<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: OCR of Hand-written Data using SVM</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../d6/de2/tutorial_py_table_of_contents_ml.html">Machine Learning</a></li><li class="navelem"><a class="el" href="../../d3/d02/tutorial_py_svm_index.html">Support Vector Machines (SVM)</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">OCR of Hand-written Data using SVM </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this chapter</p>
<ul>
<li>We will revisit the hand-written data OCR, but, with SVM instead of kNN.</li>
</ul>
<h2>OCR of Hand-written Digits </h2>
<p>In kNN, we directly used pixel intensity as the feature vector. This time we will use <a href="http://en.wikipedia.org/wiki/Histogram_of_oriented_gradients">Histogram of Oriented Gradients</a> (HOG) as feature vectors.</p>
<p>Here, before finding the HOG, we deskew the image using its second order moments. So we first define a function <b>deskew()</b> which takes a digit image and deskew it. Below is the deskew() function:</p>
<div class="fragment"><div class="line"><span class="keyword">def </span>deskew(img):</div><div class="line">    m = <a class="code" href="../../d3/dc0/group__imgproc__shape.html#ga556a180f43cab22649c23ada36a8a139">cv.moments</a>(img)</div><div class="line">    <span class="keywordflow">if</span> <a class="code" href="../../d1/d10/classcv_1_1MatExpr.html#a30843fc6c148a00f5d300a7f43f3fbdc">abs</a>(m[<span class="stringliteral">&#39;mu02&#39;</span>]) &lt; 1e-2:</div><div class="line">        <span class="keywordflow">return</span> img.copy()</div><div class="line">    skew = m[<span class="stringliteral">&#39;mu11&#39;</span>]/m[<span class="stringliteral">&#39;mu02&#39;</span>]</div><div class="line">    M = np.float32([[1, skew, -0.5*SZ*skew], [0, 1, 0]])</div><div class="line">    img = <a class="code" href="../../da/d54/group__imgproc__transform.html#ga0203d9ee5fcd28d40dbc4a1ea4451983">cv.warpAffine</a>(img,M,(SZ, SZ),flags=affine_flags)</div><div class="line">    <span class="keywordflow">return</span> img</div></div><!-- fragment --><p> Below image shows above deskew function applied to an image of zero. Left image is the original image and right image is the deskewed image.</p>
<div class="image">
<img src="../../deskew.jpg" alt="deskew.jpg"/>
<div class="caption">
image</div></div>
<p> Next we have to find the HOG Descriptor of each cell. For that, we find Sobel derivatives of each cell in X and Y direction. Then find their magnitude and direction of gradient at each pixel. This gradient is quantized to 16 integer values. Divide this image to four sub-squares. For each sub-square, calculate the histogram of direction (16 bins) weighted with their magnitude. So each sub-square gives you a vector containing 16 values. Four such vectors (of four sub-squares) together gives us a feature vector containing 64 values. This is the feature vector we use to train our data.</p>
<div class="fragment"><div class="line"><span class="keyword">def </span>hog(img):</div><div class="line">    gx = <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">cv.Sobel</a>(img, cv.CV_32F, 1, 0)</div><div class="line">    gy = <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">cv.Sobel</a>(img, cv.CV_32F, 0, 1)</div><div class="line">    mag, ang = <a class="code" href="../../d2/de8/group__core__array.html#gac5f92f48ec32cacf5275969c33ee837d">cv.cartToPolar</a>(gx, gy)</div><div class="line">    bins = np.int32(bin_n*ang/(2*np.pi))    <span class="comment"># quantizing binvalues in (0...16)</span></div><div class="line">    bin_cells = bins[:10,:10], bins[10:,:10], bins[:10,10:], bins[10:,10:]</div><div class="line">    mag_cells = mag[:10,:10], mag[10:,:10], mag[:10,10:], mag[10:,10:]</div><div class="line">    hists = [np.bincount(b.ravel(), m.ravel(), bin_n) <span class="keywordflow">for</span> b, m <span class="keywordflow">in</span> zip(bin_cells, mag_cells)]</div><div class="line">    hist = np.hstack(hists)     <span class="comment"># hist is a 64 bit vector</span></div><div class="line">    <span class="keywordflow">return</span> hist</div></div><!-- fragment --><p> Finally, as in the previous case, we start by splitting our big dataset into individual cells. For every digit, 250 cells are reserved for training data and remaining 250 data is reserved for testing. Full code is given below, you also can download it from <a href="https://github.com/opencv/opencv/tree/master/samples/python/tutorial_code/ml/py_svm_opencv/hogsvm.py">here</a>:</p>
<div class="fragment"><div class="line"><span class="comment">#!/usr/bin/env python</span></div><div class="line"></div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"></div><div class="line">SZ=20</div><div class="line">bin_n = 16 <span class="comment"># Number of bins</span></div><div class="line"></div><div class="line"></div><div class="line">affine_flags = cv.WARP_INVERSE_MAP|cv.INTER_LINEAR</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">def </span>deskew(img):</div><div class="line">    m = <a class="code" href="../../d3/dc0/group__imgproc__shape.html#ga556a180f43cab22649c23ada36a8a139">cv.moments</a>(img)</div><div class="line">    <span class="keywordflow">if</span> <a class="code" href="../../d1/d10/classcv_1_1MatExpr.html#a30843fc6c148a00f5d300a7f43f3fbdc">abs</a>(m[<span class="stringliteral">&#39;mu02&#39;</span>]) &lt; 1e-2:</div><div class="line">        <span class="keywordflow">return</span> img.copy()</div><div class="line">    skew = m[<span class="stringliteral">&#39;mu11&#39;</span>]/m[<span class="stringliteral">&#39;mu02&#39;</span>]</div><div class="line">    M = np.float32([[1, skew, -0.5*SZ*skew], [0, 1, 0]])</div><div class="line">    img = <a class="code" href="../../da/d54/group__imgproc__transform.html#ga0203d9ee5fcd28d40dbc4a1ea4451983">cv.warpAffine</a>(img,M,(SZ, SZ),flags=affine_flags)</div><div class="line">    <span class="keywordflow">return</span> img</div><div class="line"></div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">def </span>hog(img):</div><div class="line">    gx = <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">cv.Sobel</a>(img, cv.CV_32F, 1, 0)</div><div class="line">    gy = <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">cv.Sobel</a>(img, cv.CV_32F, 0, 1)</div><div class="line">    mag, ang = <a class="code" href="../../d2/de8/group__core__array.html#gac5f92f48ec32cacf5275969c33ee837d">cv.cartToPolar</a>(gx, gy)</div><div class="line">    bins = np.int32(bin_n*ang/(2*np.pi))    <span class="comment"># quantizing binvalues in (0...16)</span></div><div class="line">    bin_cells = bins[:10,:10], bins[10:,:10], bins[:10,10:], bins[10:,10:]</div><div class="line">    mag_cells = mag[:10,:10], mag[10:,:10], mag[:10,10:], mag[10:,10:]</div><div class="line">    hists = [np.bincount(b.ravel(), m.ravel(), bin_n) <span class="keywordflow">for</span> b, m <span class="keywordflow">in</span> zip(bin_cells, mag_cells)]</div><div class="line">    hist = np.hstack(hists)     <span class="comment"># hist is a 64 bit vector</span></div><div class="line">    <span class="keywordflow">return</span> hist</div><div class="line"></div><div class="line"></div><div class="line">img = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<a class="code" href="../../d6/dba/group__core__utils__samples.html#ga3a33b00033b46c698ff6340d95569c13">cv.samples.findFile</a>(<span class="stringliteral">&#39;digits.png&#39;</span>),0)</div><div class="line"><span class="keywordflow">if</span> img <span class="keywordflow">is</span> <span class="keywordtype">None</span>:</div><div class="line">    <span class="keywordflow">raise</span> Exception(<span class="stringliteral">&quot;we need the digits.png image from samples/data here !&quot;</span>)</div><div class="line"></div><div class="line"></div><div class="line">cells = [np.hsplit(row,100) <span class="keywordflow">for</span> row <span class="keywordflow">in</span> np.vsplit(img,50)]</div><div class="line"></div><div class="line"><span class="comment"># First half is trainData, remaining is testData</span></div><div class="line">train_cells = [ i[:50] <span class="keywordflow">for</span> i <span class="keywordflow">in</span> cells ]</div><div class="line">test_cells = [ i[50:] <span class="keywordflow">for</span> i <span class="keywordflow">in</span> cells]</div><div class="line"></div><div class="line"></div><div class="line"></div><div class="line">deskewed = [list(map(deskew,row)) <span class="keywordflow">for</span> row <span class="keywordflow">in</span> train_cells]</div><div class="line">hogdata = [list(map(hog,row)) <span class="keywordflow">for</span> row <span class="keywordflow">in</span> deskewed]</div><div class="line">trainData = np.float32(hogdata).reshape(-1,64)</div><div class="line">responses = np.repeat(np.arange(10),250)[:,np.newaxis]</div><div class="line"></div><div class="line">svm = cv.ml.SVM_create()</div><div class="line">svm.setKernel(cv.ml.SVM_LINEAR)</div><div class="line">svm.setType(cv.ml.SVM_C_SVC)</div><div class="line">svm.setC(2.67)</div><div class="line">svm.setGamma(5.383)</div><div class="line"></div><div class="line">svm.train(trainData, cv.ml.ROW_SAMPLE, responses)</div><div class="line">svm.save(<span class="stringliteral">&#39;svm_data.dat&#39;</span>)</div><div class="line"></div><div class="line"></div><div class="line"></div><div class="line">deskewed = [list(map(deskew,row)) <span class="keywordflow">for</span> row <span class="keywordflow">in</span> test_cells]</div><div class="line">hogdata = [list(map(hog,row)) <span class="keywordflow">for</span> row <span class="keywordflow">in</span> deskewed]</div><div class="line">testData = np.float32(hogdata).reshape(-1,bin_n*4)</div><div class="line">result = svm.predict(testData)[1]</div><div class="line"></div><div class="line"></div><div class="line">mask = result==responses</div><div class="line">correct = np.count_nonzero(mask)</div><div class="line"><a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>(correct*100.0/result.size)</div></div><!-- fragment --><p>This particular technique gave me nearly 94% accuracy. You can try different values for various parameters of SVM to check if higher accuracy is possible. Or you can read technical papers on this area and try to implement them.</p>
<h2>Additional Resources </h2>
<ol type="1">
<li><a href="https://www.youtube.com/watch?v=0Zib1YEE4LU">Histograms of Oriented Gradients Video</a></li>
</ol>
<h2>Exercises </h2>
<ol type="1">
<li>OpenCV samples contain digits.py which applies a slight improvement of the above method to get improved result. It also contains the reference. Check it and understand it. </li>
</ol>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
